Method and apparatus for providing user input back channel in audio/video system

ABSTRACT

A method and apparatus for providing a user input back channel (UIBC) in an audio/video (AV) source device and an AV sink device communicating according to a wireless fidelity (Wi-Fi) display (WFD) standard is provided. The method includes: setting up an AV control session and an AV data session between the AV source devices according to the WFD standards; enabling the UIBC from the AV sink device to the AV source device by using the AV control session; and transmitting a user input from the AV sink device to the AV source device through the UIBC.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application claims priority from U.S. Provisional Application No.61/257,186, filed on Nov. 2, 2009, in the U.S. Patent and TrademarkOffice, and Korean Patent Application No. 10-2010-0101872, filed on Oct.19, 2010, in the Korean Intellectual Property Office, the disclosures ofwhich are incorporated herein in their entirety by reference.

BACKGROUND

1. Field

Apparatuses and methods consistent with the exemplary embodiments relateto providing a user input back channel (UIBC) in an audio/video (AV)system, and more particularly, to a method and apparatus for providing aUIBC in an AV source device and an AV sink device communicatingaccording to a wireless fidelity (Wi-Fi) display (WFD) standard.

2. Description of the Related Art

Recently, a wireless fidelity (Wi-Fi) display (WFD) standard has beennewly defined based on a requirement to transmit audio/video (AV) datawhile satisfying high quality and low latency. The WFD standardconsiders AV data transmission and control via a user input in a sourcedevice. However, it is highly likely that a user may be nearer to a sinkdevice than the source device, and thus when the user controls a deviceor inputs data via the sink device, a method of transmitting a controlcommand or the user data from the sink device to the source device isrequired.

SUMMARY

The exemplary embodiments provide a user input back channel (UIBC)between an audio/video (AV) source device and an AV sink devicecommunicating according to a wireless fidelity (Wi-Fi) display (WFD)standard.

According to an aspect of the exemplary embodiments, there is provided amethod of communicating with an audio/video (AV) source device, whereinthe method is performed by an AV sink device, the method including:setting up an AV control session and an AV data session with the AVsource device according to a wireless fidelity (Wi-Fi) display (WFD)standard; enabling a user input back channel (UIBC) to the AV sourcedevice, by using the AV control session; and transmitting a user inputto the AV source device through the UIBC.

The user input may include at least one of a function code forcontrolling the AV source device, pointing information, and characterdata.

The AV control session may be a real-time streaming protocol (RTSP)session.

The enabling of the UIBC may include: receiving a query about the UIBCcapability of the AV sink device from the AV source device; transmittinga response related to the UIBC capability of the AV sink device to theAV source device; receiving, from the AV source device, a UIBC enablerequest based on the UIBC capability of the AV sink device; andtransmitting a response related to the UIBC enable request to the AVsource device.

The UIBC capability of the AV sink device may include at least one ofRTSP, real-time transport protocol (RTP), Bluetooth, Zigbee, and digitalliving network association (DLNA).

The enabling of the UIBC may include: transmitting a query about theUIBC capability of the AV source device to the AV source device;receiving a response related to the UIBC capability of the AV sourcedevice from the AV source device; transmitting a UIBC enable request tothe AV source device based on the UIBC capability of the AV sourcedevice; and receiving a response related to the UIBC enable request fromthe AV source device.

The UIBC capability of the AV source device may include at least one ofreal-time streaming protocol (RTSP), real-time transport protocol (RTP),Bluetooth, Zigbee, and digital living network association (DLNA).

The transmitting of the user input may include transmitting data andcontrol information about the user input through the AV control session.

The enabling of the UIBC may include setting up a data session of theUIBC by using the AV control session, and the transmitting the userinput may include transmitting data about the user input through thedata session of the UIBC.

The transmitting the user input may further include transmitting controlinformation about the user input through the AV control session.

The data session of the UIBC may include at least one of an RTP session,a Bluetooth session, a Zigbee session, and a DLNA session.

The transmitting the user input may further include transmitting controlinformation about the user input through the DLNA session.

The method may further include notifying disablement of the UIBC to theAV source device.

According to another aspect of the exemplary embodiments, there isprovided a method of communicating with an audio/video (AV) sink device,wherein the method is performed by an AV source device, the methodincluding: setting up an AV control session and an AV data session withthe AV sink device according to a wireless fidelity (Wi-Fi) display(WFD) standard; enabling a user input back channel (UIBC) from the AVsink device, by using the AV control session; and receiving a user inputfrom the AV sink device through the UIBC.

The user input may include at least one of a function code forcontrolling the AV source device, pointing information, and characterdata.

The AV control session may be a real-time streaming protocol (RTSP)session.

The enabling of the UIBC may include: transmitting a query about theUIBC capability of the AV sink device to the AV sink device; receivingabout a response related to the UIBC capability of the AV sink devicefrom the AV sink device; transmitting a UIBC enable request to the AVsink device based on the UIBC capability of the AV sink device; andreceiving a response related to the UIBC enable request from the AV sinkdevice.

The UIBC capability of the AV sink device may include at least one ofRTSP, real-time transport protocol (RTP), Bluetooth, Zigbee, and digitalliving network association (DLNA).

The enabling of the UIBC may include: receiving a query about UIBCcapability of the AV source device from the AV sink device; transmittingabout a response related to the UIBC capability of the AV source deviceto the AV sink device; receiving a UIBC enable request from the AV sinkdevice based on the UIBC capability of the AV source device; andtransmitting a response related to the UIBC enable request to the AVsink device.

The UIBC capability of the AV source device may include at least one ofRTSP, RTP, Bluetooth, Zigbee, and DLNA.

The receiving the user input may include receiving data and controlinformation about the user input through the AV control session.

The enabling of the UIBC may include setting up a data session of theUIBC by using the AV control session, and the receiving the user inputmay include receiving data bout the user input through the data sessionof the UIBC.

The receiving the user input may further include receiving controlinformation about the user input through the AV control session.

The data session of the UIBC may include at least one of a RTP session,a Bluetooth session, a Zigbee session, and a DLNA session.

The receiving the user input may further include receiving controlinformation about the user input through the DLNA session.

The method may further include: transmitting a UIBC disable request tothe AV sink device; and receiving a response about the UIBC disablerequest from the AV sink device.

According to another aspect of the exemplary embodiments, there isprovided an audio/video (AV) sink device communicating with an AV sourcedevice, the AV sink device including: a controller for setting up an AVcontrol session and an AV data session with the AV source deviceaccording to a wireless fidelity (Wi-Fi) display (WFD) standard, andenabling a user input back channel (UIBC) to the AV source device byusing the AV control session; and a communicator for transmitting a userinput to the AV source device through the UIBC.

The communicator may transmit data and control information about theuser input through the AV control session.

The controller may set up a data session of the UIBC by using the AVcontrol session, and the communicator may transmit data about the userinput through the data session of the UIBC.

The communicator may transmit control information about the user inputthrough the AV control session.

According to another aspect of the exemplary embodiments, there isprovided an audio/video (AV) source device communicating with an AV sinkdevice, the AV source device including: a controller for setting up anAV control session and an AV data session with the AV sink deviceaccording to a wireless fidelity (Wi-Fi) display (WFD) standard, andenabling a user input back channel (UIBC) from the AV sink device byusing the AV control session; and a communicator for receiving a userinput from the AV sink device through the UIBC.

The communicator may receive data and control information about the userinput through the AV control session.

The controller may set up a data session of the UIBC by using the AVcontrol session, and the communicator may receive data about the userinput through the data session of the UIBC.

The communicator may receive control information about the user inputthrough the AV control session.

According to another aspect of the exemplary embodiments, there isprovided a communication system including: an audio/video (AV) sourcedevice transmitting AV data; and an AV sink device receiving AV data,wherein the AV source device sets up an AV control session and an AVdata session with the AV sink device, according to a wireless fidelity(Wi-Fi) display (WFD) standard, and enables a user input back channel(UIBC) from the AV sink device by using the AV control session, and theAV sink device transmits a user input to the AV source device throughthe UIBC.

According to another aspect of the exemplary embodiments, there isprovided a computer readable recording medium having embodied thereon,instructions that, when executed by a computer, causes the computer toperform any one of the above methods.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects will become more apparent by describing indetail exemplary embodiments thereof with reference to the attacheddrawings in which:

FIG. 1 is a diagram of a protocol stack of a wireless fidelity (Wi-Fi)display (WFD) standard, according to an exemplary embodiment;

FIG. 2 is a diagram schematically illustrating a flow for generating aWFD audio/video (AV) session according to an exemplary embodiment;

FIG. 3 is a diagram of a use case of a WFD, according to an exemplaryembodiment;

FIG. 4 is a diagram of a use case of a WFD, according to anotherexemplary embodiment;

FIG. 5 is a diagram for describing a user input back channel (UIBC)according to an exemplary embodiment;

FIG. 6 is a flowchart schematically illustrating a method of providing aUIBC, according to an exemplary embodiment;

FIG. 7 is a diagram of a protocol stack of an in-band UIBC according toan exemplary embodiment;

FIG. 8 is a diagram of a protocol stack of an out-of-band UIBC using areal-time transport protocol (RTP), according to an exemplaryembodiment;

FIGS. 9 and 10 are diagrams of a protocol stack of an out-of-band UIBCusing a separate protocol, according to exemplary embodiments;

FIG. 11 is a diagram of query and response message formats about UIBCcapability, according to an exemplary embodiment;

FIG. 12 is a diagram of a definition of a wfd_uibc_capability parameteraccording to an exemplary embodiment;

FIG. 13 is a diagram of UIBC enable request and response message formatsaccording to an exemplary embodiment;

FIG. 14 is a diagram of a definition of a wfd_uibc_setting parameteraccording to an exemplary embodiment;

FIG. 15 is a diagram of a user input data message format using in-bandUIBC, according to an exemplary embodiment;

FIG. 16 is a diagram of a definition of a wfd_uibc_user_data parameteraccording to an exemplary embodiment;

FIG. 17 is a diagram of a UIBC disable request message format accordingto an exemplary embodiment;

FIG. 18 is a diagram of a UIBC disable notification message formataccording to an exemplary embodiment;

FIG. 19 is a flowchart schematically illustrating a method ofcommunicating with an AV source device, wherein the method is performedby an AV sink device, according to an exemplary embodiment;

FIG. 20 is a flowchart schematically illustrating a method ofcommunicating with an AV sink device, wherein the method is performed byan AV source device, according to an exemplary embodiment; and

FIG. 21 is a block diagram schematically illustrating an AVcommunication system according to an exemplary embodiment.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Hereinafter, the exemplary embodiments will be described more fully withreference to the accompanying drawings, in which exemplary embodimentsare shown. The exemplary embodiments may, however, be embodied in manydifferent forms and should not be construed as being limited to theexemplary embodiments set forth herein; rather, these exemplaryembodiments are provided so that this disclosure will be thorough andcomplete, and will fully convey the concept of the exemplary embodimentsto those skilled in the art.

While describing the exemplary embodiments, detailed descriptions aboutrelated well-known functions or configurations that may diminish theclarity of the points of the exemplary embodiments are omitted. Also,the terminologies used are defined considering functions herein, and maybe called differently based on intentions or customs of a user or anoperator. Accordingly, the terminologies may be defined based on thecontents of the specification. Also, when a part “includes” an element,the part may further include another element, unless otherwisedescribed. For convenience of description, an apparatus and a methodwill be described together if required.

In the drawings, like reference numerals denote like elements, and thesizes and thicknesses of elements may be exaggerated for clarity.

FIG. 1 is a diagram of a protocol stack of a wireless fidelity (Wi-Fi)display (WFD) standard, according to an exemplary embodiment. FIG. 1shows a protocol stack of layer 3 (L3) internet protocol (IP) layer andupper, wherein a protocol of layer 2 (L2) or lower may use Wi-Fi peer topeer (P2P) standard or other standards defined in Wi-Fi. A protocolstack of WFD is largely classified into a control path and a data path.According to an exemplary embodiment, a real-time streaming protocol(RTSP) may be used as the control path, and a real-time transportprotocol (RTP) or an RTP control protocol (RTCP) may be used as the datapath. The RTSP, the RTP, and the RTCP are well known to one of ordinaryskill in the art, and thus will not be described in detail asdescriptions thereof may diminish the clarity of the points of theexemplary embodiments.

FIG. 2 is a diagram schematically illustrating a flow for generating aWFD audio/video (AV) session according to an exemplary embodiment.First, L2 device and service discovery is performed in operation 210,between a source device and a sink device supporting a WFD. At thistime, it is determined that the source device and the sink devicesupport the WFD. Here, an L2 secure setup is performed in operation 220,and then L3 WFD discovery, in which detailed device information andservice information is exchanged, is performed in operation 230. Then,an L3 AV session is established in operation 240. Here, an RTSP is usedfor an AV control session, an RTP is used for an AV data session, and ahigh-bandwidth digital content protection (HDCP) 2.0 is used for contentprotection. AV data is converted into a data unit transportableaccording to MPEG2 TS or H.264, and is transmitted from the sourcedevice to the sink device via the RTP. An error feedback is performed byusing the RTCP. After the AV data is transmitted, an AV session isended.

Here, when a user input, such as a control command or data, is input tothe source device, the user input is directly reflected on the sourcedevice. In other words, when the user input is input to the sourcedevice, the source device processes the user input without having totransmit the user input to the sink device.

FIG. 3 is a diagram of an example of a WFD, according to an exemplaryembodiment. Referring to FIG. 3, a personal computer (PC) 310 in abedroom transmits AV data to a television (TV) 320 in a living room, andthe TV 320 outputs the AV data upon receiving the AV data. Thus, a usermay view AV contents provided by the PC 310 in the bedroom through theTV 320 in the living room. Here, the PC 310 transmitting the AV data isan AV source device, and the TV 320 receiving the AV data is an AV sinkdevice.

Since the user is nearer to the TV 320 than the PC 310, the user maywant to control the system by using an input device of the TV 320, suchas a remote controller 330, a mouse 340, or a keyboard 350.

FIG. 4 is a diagram of a use case of a WFD, according to anotherexemplary embodiment. Referring to FIG. 4, a mobile device 410, such asa smart phone or a mobile internet device (MID), provides AV data, andsince a screen of the mobile device 410 is small, a user may want toview the AV data through a lap top 420 having a larger screen than themobile device 410. Here, since an input device of the lap top 420 isalso larger and more convenient than that of the mobile device 410, theuser may want to control the system by using the input device of the laptop 420. Here, the mobile device 410 transmitting the AV data is an AVsource device, and the lap top 420 receiving the AV data is an AV sinkdevice.

In the illustrations shown in FIGS. 3 and 4, since AV data is providedby a source device and a user input is input in a sink device, it isrequired to transmit the user input from the sink device to the sourcedevice. Here, the user input includes function code or command messageinformation generated through a function key of a remote control,pointing coordinate information generated by a pointing device, such asa mouse or a touch panel, or a character or text generated by akeyboard. The function code or command message information, the pointingcoordinate information, and the character or text information areprocessed according to corresponding characteristics.

FIG. 5 is a diagram for describing a user input back channel (UIBC)according to an exemplary embodiment. Methods of transmitting a userinput from a sink device to a source device may be largely classifiedinto an in-band UIBC (IB-UIBC) method and an out-of-band UIBC OOB-UIBC)method. The IB-UIBC method transmits the user input by using a relatedart SET_PARAMETER method by adding new parameters for the user input toa related art RTSP. In the OOB-UIBC method, setup or control of the UIBCis implemented by adding new parameters to an RTSP, but the user inputuses a separate RTP session or is transmitted through a separateprotocol, such as Bluetooth, Zigbee, or digital living networkassociation (DLNA). Also, as described above, the user input may be afunction code, pointing information, or character data, wherein varioustypes of user inputs may be transmitted through one UIBC or differentUIBCs.

FIG. 6 is a flowchart schematically illustrating a method of providing aUIBC, according to an exemplary embodiment. A source device and a sinkdevice set up an AV control session and an AV data session according toa WFD standard, and then a UIBC from the sink device to the sourcedevice is enabled by using the AV control session, in operations 510through 540.

First, the source device queries the UIBC capability of the sink device,in operation 510. Here, a wfd_uibc_capability parameter indicating theUIBC capability is defined, and is queried according to GET_PARAMETER.The sink device responds about its UIBC capability in operation 520. TheUIBC capability may be included as an information element or field inthe device capability or sink capability defined in the WFD standard.

The source device requests the sink device to enable a desired UIBC fromamong UIBCs provided by the sink device, according to a user selectionor an automatic selection, in operation 530. Such a UIBC enable requestmessage uses a SET_PARAMETER message in an RTSP, and newly defines arelated parameter. The parameter related to the UIBC enable requestmessage is a wfd_uibc_setting parameter, and the source device mayenable or disable the desired parameter by using the wfd_uibc_settingparameter. Definition of the wfd_uibc_setting parameter will bedescribed later. Upon receiving the UIBC enable request message, thesink device enables the desired UIBC, and responds to the source deviceabout a result of enablement, in operation 540.

Such a process of enabling a UIBC described above may be performedaccording to a request of the sink device. In other words, operations510 and 530 may be performed by the sink device and operations 520 and540 may be performed by the source device, thereby enabling a UIBC fromthe sink device to the source device.

When the desired UIBC is enabled, a user input generated in the sinkdevice is transmitted to the source device in operations 551, 552, 553,and 560, and thus AV data or a WFD function may be controlled. In theIB-UIBC method, a wfd_uibc_user_data parameter is transmitted to thesource device by using the SET_PARAMETER, and in the OOB-UIBC, the userinput may be transmitted through data payload.

When the UIBC function is disabled by the source device, the sourcedevice transmits a disable UIBC request message to the sink device inoperation 570, and the sink device responds to the source device afterdisabling the UIBC function in operation 580. When the sink devicedisables the UIBC, the sink device may notify the source device that theUIBC is disabled by using a method of transmitting the wfd_uibc_settingparameter to the source device by using the SET_PARAMETER, in operation590.

FIG. 7 is a diagram of a protocol stack of an IB-UIBC according to anexemplary embodiment. Referring to FIG. 7, the IB-UIBC uses an RTSP as adata path and a control path. In other words, data and controlinformation about a user input are all transmitted through the RTSP. TheIB-UIBC is realized by additionally defining new parameters fortransmitting the user input, in an RTSP message. The added parameter istransmitted to a source device through a SET_PARAMETER. The addedparameter, i.e., a wfd_uibc_user_data parameter, includes informationabout a user input type, and a user input data message is definedaccording to a corresponding type. Definition of the wfd_uibc_user_dataparameter will be described later. Upon receiving the user input from asink device, the source device responds about the reception.

In an OOB-UIBC method, the user input is transmitted by using a separateRTP session or through a separate path, such as Bluetooth, Zigbee, orDLNA, instead of transmitting the user input through an existing RTSP.In the OOB-UIBC method, a control path for requesting UIBC capability orenablement of a UIBC may use an RTSP, like in an IB-UIBC method.According to another exemplary embodiment, a UIBC may be controlledthrough a separate path, like the user input.

FIG. 8 is a diagram of a protocol stack of an OOB-UIBC using a RTP,according to an exemplary embodiment. Referring to FIG. 8, a data pathof a UIBC uses a separate RTP session, and a control path of the UIBCuses an RTSP. In other words, the separate RTP session is set up betweena source device and a sink device so as to be used as a path fortransmitting a user input. A user input data message may be directlyinserted into and transmitted with RTP payload, or coded as andtransmitted with a text/t140 block by using RTP payload for textconversation (RFC4103). Here, text/t140 is a multipurpose internal mailextensions (MIME) registered type, is a format defined in RFC4103, isUTF-8 coded, and used in H.323, SIP, 3GPP, or the like.

FIGS. 9 and 10 are diagrams of a protocol stack of an OOB-UIBC using aseparate protocol, according to exemplary embodiments.

FIG. 9 is a diagram of a protocol stack of an OOB-UIBC using Bluetoothor Zigbee, according to an exemplary embodiment. Referring to FIG. 9, adata path of a UIBC uses Bluetooth or Zigbee, and a control path of theUIBC uses an RTSP. Alternatively, the control path of the UIBC may useBluetooth or Zigbee.

The OOB-UIBC using a separate protocol may be used to transmit a userinput in real-time while minimizing an affect between AV datatransmission and user input transmission, and may be used when the userinput requires low latency, like pointing information, or when the userinput is to be transmitted while not effecting AV data streaming. Here,a source device and a sink device support the same protocol. The userinput is transmitted to application data of the separate protocol. Auser input data message may be directly inserted into and transmittedwith a payload, or only the user input may be inserted into andtransmitted with a payload.

FIG. 10 is a diagram of a protocol stack of an OOB-UIBC using a DLNA.Referring to FIG. 9, a data path of a UIBC uses the DLNA, and a controlpath of the UIBC uses an RTSP. Alternatively, the control path of theUIBC may use the DLNA. A user input may be transmitted to a remote userinterface (RUI) of the DLAN, and a payload may use a format defined inthe DLNA.

UIBC capability is exchanged or UIBC is enabled while the RTSP is set upfor control or management, in a WFD specification. Formats of messagesand definitions of parameters described above will now be described.

FIG. 11 is a diagram of query and response message formats about UIBCcapability, according to an exemplary embodiment. Referring to FIG. 11,in order for a source device to transmit a UIBC capability query to asink device, the source device requests a wfd_uibc_capability parameterby using a GET_PARAMETER method. A response of the sink device about theUIBC capability query follows a response format of the GET_PARAMETERdefined in an RTSP. Referring to FIG. 11, the sink device sends aresponse to the source device indicating that the sink device supportsan IB-UIBC and an OOB-UIBC using an RTP, and has the capability oftransmitting a function code through the IB-UIBC and pointinginformation through the IB-UIBC or OOB-UIBC.

FIG. 12 is a diagram of a definition of a wfd_uibc_capability parameteraccording to an exemplary embodiment. Referring to FIG. 12, thewfd_uibc_capability parameter may be defined in an augmented Backus-NaurForm (ABNF) format, and includes information about whether a UIBC issupported, a supported UIBC path, and a supported user input data type.A UIBC_detailed_capa parameter may be defined considering that asupported UIBC path may be different according to a user input type.

FIG. 13 is a diagram of UIBC enable request and response message formatsaccording to an exemplary embodiment. Referring to FIG. 13, when asource device transmits a request to a sink device to enable a UIBC, awfd_uibc_setting parameter is set and transmitted to the sink device byusing a SET_PARAMETER method. In FIG. 13, the source device enables auser input related to a function code and pointing information of thesink device, and uses only an IB_UIBC as a path.

A response by the sink device to the request made by the AV sourcedevice to enable the UIBC follows a response format of a SET_PARAMETERdefined in an RTSP. The sink device responds with an error code if therequest by the source device is not processed.

FIG. 14 is a diagram of a definition of a wfd_uibc_setting parameteraccording to an exemplary embodiment. Referring to FIG. 14, thewfd_uibc_setting parameter defines a UIBC path to be enabled by a sinkdevice, and a user input data type. A detailed_setting_en parameter maybe defined considering that a supported UIBC path may be differentaccording to a type of user input.

FIG. 15 is a diagram of a user input data message format using IB-UIBC,according to an exemplary embodiment. Referring to FIG. 15, aSET_PARAMETER method is used when a user input is transmitted from asink device to a source device by using an IB-UIBC path. Here, awfd_uibc_user_data parameter is used.

A response of the sink device to the user input follows a responseformat of the SET_PARAMETER defined in an RTSP. The source deviceresponds with an error code if the request by the sink device is notprocessed.

FIG. 16 is a diagram of a definition of a wfd_uibc_user_data parameteraccording to an exemplary embodiment. Referring to FIG. 16, thewfd_uibc_user_data parameter defines a user input type and user inputdata input to a sink device. The same parameter is used when a userinput is transmitted through an OOB-UIBC. When the user input data is afunction code, the user input data is defined on an octet. When the userinput data is pointing information, the wfd_uibc_user_data parameter isdefined in a coordinate of a current point, resolution of the sinkdevice, and an action of a user. The action of the user may show whethera user clicked a button, and which button is clicked. When the userinput data is a character, CHAR is input.

FIG. 17 is a diagram of a UIBC disable request message format accordingto an exemplary embodiment. Referring to FIG. 17, when a source devicerequests to disable a UIBC, a wfd_uibc_setting parameter is disabled andtransmitted to a sink device by using a SET_PARAMETER method. Thewfd_uibc_setting is also used when multiple UIBC paths or types areenabled and a change is required.

A response by the sink device to the request by the AV source device todisable the UIBC follows a response format of a SET_PARAMETER defined inan RTSP. The sink device responds with an error code if the request bythe source device is not processed.

FIG. 18 is a diagram of a UIBC disable notification message formataccording to an exemplary embodiment. Referring to FIG. 18, awfd_uibc_setting parameter is disabled and transmitted to a sourcedevice by using a SET_PARAMETER method when a UIBC is disabled due toselection of a user through a sink device or other reasons. Here, thesource device may not respond.

FIG. 19 is a flowchart schematically illustrating a method ofcommunicating with an AV source device, wherein the method is performedby an AV sink device, according to an exemplary embodiment. Referring toFIG. 19, the method includes setting up an AV control session and an AVdata session with an AV source device, according to a WFD standard inoperation 1910, enabling a UIBC to the AV source device by using the AVcontrol session in operation 1920, and transmitting a user input to theAV source device through the UIBC in operation 1930.

FIG. 20 is a flowchart schematically illustrating a method ofcommunicating with an AV sink device, wherein the method is performed byan AV source device, according to an exemplary embodiment. Referring toFIG. 20, the method includes setting up an AV control session and an AVdata session with an AV sink device, according to a WFD standard inoperation 2010, enabling a UIBC from the AV sink device by using the AVcontrol session in operation 2020, and receiving a user input from theAV sink device through the UIBC in operation 2030.

FIG. 21 is a block diagram schematically illustrating an AVcommunication system according to an exemplary embodiment. Referring toFIG. 21, the AV communication system includes an AV source device 2110and an AV sink device 2140. The AV source device 2110 includes acontroller 2120 for setting up an AV control session and an AV datasession with the AV sink device 2140 according to a WFD standard, andenabling a UIBC from the AV sink device 2140 by using the AV controlsession, and a communicator 2130 for receiving a user input from the AVsink device 2140 through the UIBC. The AV sink device 2140 includes acontroller 2160 for setting up an AV control session and an AV datasession with the AV source device 2110 according to the WFD standard,and enabling a UIBC to the AV source device 2110 by using the AV controlsession, and a communicator 2150 for transmitting a user input to the AVsource device 2110 through the UIBC.

As described above, a method of transmitting a user input from a sinkdevice to a source device in a WFD standard is provided. Also, userconvenience and expansibility are increased as a UIBC can be set upaccording to data paths or user data types. Also, the UIBC is controlledby adding new parameters to a related art RTSP, and thus the method iseasily realized.

As is well known to one of ordinary skill in the art, the exemplaryembodiments may be variously realized, such as in a program commandexecuted by a processor, a software module, a microcode, a computerprogram product recorded on a computer (including all devices having aninformation processing function) readable recording medium, a logiccircuit, a semiconductor on demand, or firmware. Also, the exemplaryembodiments may be realized in hardware, software, or in a combinationof hardware and software.

The computer readable recording medium is any data storage device thatcan store data which can be thereafter read by a computer system.Examples of the computer readable recording medium include read-onlymemory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes,floppy disks, optical data storage devices, etc. The computer readablerecording medium can also be distributed over network coupled computersystems so that the computer readable code is stored and executed in adistributed fashion. Also, functional programs, codes, and code segmentsfor accomplishing the exemplary embodiments can be easily construed byprogrammers skilled in the art to which the exemplary embodimentspertain. Also, a data format of a message used herein may be recorded ona recording medium.

While the aspects have been particularly shown and described withreference to exemplary embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade therein without departing from the spirit and scope of theexemplary embodiments as defined by the appended claims. The exemplaryembodiments should be considered in descriptive sense only and not forpurposes of limitation. Therefore, the scope of the exemplaryembodiments is defined not by the detailed description of the exemplaryembodiments, but by the appended claims, and all differences within thescope will be construed as being included in the exemplary embodiments.

While the aspects have been particularly shown and described withreference to exemplary embodiments thereof, it will be understood bythose of ordinary skill in the art that various changes in form anddetails may be made therein without departing from the spirit and scopeof the exemplary embodiments as defined by the following claims.

1. A method of communicating with an audio/video (AV) source device,wherein the method is performed by an AV sink device, the methodcomprising: setting up, by the AV sink device, an AV control session andan AV data session with the AV source device according to a wirelessfidelity (Wi-Fi) display (WFD) standard; enabling a user input backchannel (UIBC) to the AV source device, by using the AV control session;and transmitting a user input to the AV source device through the UIBC.2. The method of claim 1, wherein the user input comprises at least oneof a function code for controlling the AV source device, pointinginformation, and character data.
 3. The method of claim 1, wherein theAV control session is a real-time streaming protocol (RTSP) session. 4.The method of claim 1, wherein the enabling the UIBC comprises:receiving a query from the AV source device related to a UIBC capabilityof the AV sink device; transmitting a response related to the UIBCcapability of the AV sink device to the AV source device; receiving aUIBC enable request from the AV source device based on the UIBCcapability of the AV sink device; and transmitting a response related tothe UIBC enable request to the AV source device.
 5. The method of claim4, wherein the UIBC capability of the AV sink device comprises at leastone of a real-time streaming protocol (RTSP), a real-time transportprotocol (RTP), Bluetooth, Zigbee, and digital living networkassociation (DLNA).
 6. The method of claim 1, wherein the enabling ofthe UIBC comprises: transmitting, by the AV sink device, a query aboutUIBC capability of the AV source device to the AV source device;receiving, from the AV source device, a response related to the UIBCcapability of the AV source device; transmitting a UIBC enable requestto the AV source device based on the UIBC capability of the AV sourcedevice; and receiving a response related to the UIBC enable request fromthe AV source device.
 7. The method of claim 6, wherein the UIBCcapability of the AV source device comprises at least one of real-timestreaming protocol (RTSP), real-time transport protocol (RTP),Bluetooth, Zigbee, and digital living network association (DLNA).
 8. Themethod of claim 1, wherein the transmitting of the user input comprisestransmitting data and control information about the user input throughthe AV control session.
 9. The method of claim 1, wherein the enablingof the UIBC comprises setting up a data session of the UIBC by using theAV control session, and the transmitting the user input comprisestransmitting data about the user input through the data session of theUIBC.
 10. The method of claim 9, wherein the transmitting of the userinput further comprises transmitting control information about the userinput through the AV control session.
 11. The method of claim 9, whereinthe data session of the UIBC comprises at least one of a real-timetransport protocol (RTP) session, a Bluetooth session, a Zigbee session,and a digital living network association (DLNA) session.
 12. The methodof claim 11, wherein the transmitting of the user input furthercomprises transmitting control information about the user input throughthe DLNA session.
 13. The method of claim 1, further comprisingnotifying disablement of the UIBC to the AV source device.
 14. A methodof communicating with an audio/video (AV) sink device, wherein themethod is performed by an AV source device, the method comprising:setting up, by the AV source device, an AV control session and an AVdata session with the AV sink device according to a wireless fidelity(Wi-Fi) display (WFD) standard; enabling a user input back channel(UIBC) from the AV sink device, by using the AV control session; andreceiving a user input from the AV sink device through the UIBC.
 15. Themethod of claim 14, wherein the user input comprises at least one of afunction code for controlling the AV source device, pointinginformation, and character data.
 16. The method of claim 14, wherein theAV control session is a real-time streaming protocol (RTSP) session. 17.The method of claim 14, wherein the enabling of the UIBC comprises:transmitting a query related to a UIBC capability of the AV sink deviceto the AV sink device; receiving about a response related to the UIBCcapability of the AV sink device from the AV sink device; transmitting aUIBC enable request to the AV sink device based on the UIBC capabilityof the AV sink device; and receiving a response related to the UIBCenable request from the AV sink device.
 18. The method of claim 17,wherein the UIBC capability of the AV sink device comprises at least oneof real-time streaming protocol (RTSP), real-time transport protocol(RTP), Bluetooth, Zigbee, and digital living network association (DLNA).19. The method of claim 14, wherein the enabling of the UIBC comprises:receiving a query related to a UIBC capability of the AV source devicefrom the AV sink device; transmitting a response related to the UIBCcapability of the AV source device to the AV sink device; receiving aUIBC enable request from the AV sink device based on the UIBC capabilityof the AV source device; and transmitting a response related to the UIBCenable request to the AV sink device.
 20. The method of claim 19,wherein the UIBC capability of the AV source device comprises at leastone of real-time streaming protocol (RTSP), real-time transport protocol(RTP), Bluetooth, Zigbee, and digital living network association (DLNA).21. The method of claim 14, wherein the receiving of the user inputcomprises receiving data and control information about the user inputthrough the AV control session.
 22. The method of claim 14, wherein theenabling of the UIBC comprises setting up a data session of the UIBC byusing the AV control session, and the receiving of the user inputcomprises receiving data related to the user input through the datasession of the UIBC.
 23. The method of claim 22, wherein the receivingof the user input further comprises receiving control information aboutthe user input through the AV control session.
 24. The method of claim22, wherein the data session of the UIBC comprises at least one of areal-time transport protocol (RTP) session, a Bluetooth session, aZigbee session, and a digital living network association (DLNA) session.25. The method of claim 24, wherein the receiving of the user inputfurther comprises receiving control information about the user inputthrough the DLNA session.
 26. The method of claim 14, furthercomprising: transmitting a UIBC disable request to the AV sink device;and receiving a response related to the UIBC disable request from the AVsink device.
 27. An audio/video (AV) sink device communicating with anAV source device, the AV sink device comprising: a controller which setsup an AV control session and an AV data session with the AV sourcedevice according to a wireless fidelity (Wi-Fi) display (WFD) standard,and enables a user input back channel (UIBC) to the AV source device byusing the AV control session; and a communicator which transmits a userinput to the AV source device through the UIBC.
 28. The AV sink deviceof claim 27, wherein the communicator transmits data and controlinformation related to the user input through the AV control session.29. The AV sink device of claim 27, wherein the controller sets up adata session of the UIBC using the AV control session, and thecommunicator transmits data related to the user input through the datasession of the UIBC.
 30. The AV sink device of claim 29, wherein thecommunicator transmits control information related to the user inputthrough the AV control session.
 31. An audio/video (AV) source devicecommunicating with an AV sink device, the AV source device comprising: acontroller which sets up an AV control session and an AV data sessionwith the AV sink device according to a wireless fidelity (Wi-Fi) display(WFD) standard, and enables a user input back channel (UIBC) from the AVsink device by using the AV control session; and a communicator whichreceives a user input from the AV sink device through the UIBC.
 32. TheAV source device of claim 31, wherein the communicator receives data andcontrol information related to the user input through the AV controlsession.
 33. The AV source device of claim 31, wherein the controllersets up a data session of the UIBC by using the AV control session, andthe communicator receives data relating to the user input through thedata session of the UIBC.
 34. The AV source device of claim 33, whereinthe communicator receives control information relating to the user inputthrough the AV control session.
 35. A communication system comprising:an audio/video (AV) source device which transmits AV data; and an AVsink device which receives AV data, wherein the AV source device sets upan AV control session and an AV data session with the AV sink device,according to a wireless fidelity (Wi-Fi) display (WFD) standard, andenables a user input back channel (UIBC) from the AV sink device byusing the AV control session, and the AV sink device transmits a userinput to the AV source device through the UIBC.
 36. A computer readablerecording medium having embodied thereon instructions, that whenexecuted by a computer, causes the computer to perform a method ofcommunicating with an audio/video (AV) source device, wherein the methodis performed by an AV sink device, the method comprising: setting up, bythe AV sink device, an AV control session and an AV data session withthe AV source device according to a wireless fidelity (Wi-Fi) display(WFD) standard; enabling a user input back channel (UIBC) to the AVsource device, by using the AV control session; and transmitting a userinput to the AV source device through the UIBC.
 37. A computer readablerecording medium having embodied thereon instructions that, whenexecuted by a computer, causes the computer to perform a method ofcommunicating with an audio/video (AV) sink device, wherein the methodis performed by an AV source device, the method comprising: setting up,by the AV source device, an AV control session and an AV data sessionwith the AV sink device according to a wireless fidelity (Wi-Fi) display(WFD) standard; enabling a user input back channel (UIBC) from the AVsink device, by using the AV control session; and receiving a user inputfrom the AV sink device through the UIBC.
 38. A method of providing auser input back channel (UIBC) in an AV source device and an AV sinkdevice, the method comprising: transmitting, by the source device, aquery related to UIBC capabilities of the AV sink device to the AV sinkdevice; receiving, by the AV source device, a response related to theUIBC capabilities of the AV sink device, transmitting, to the AV sinkdevice, a request to enable a desired UIBC, and enabling the desiredUIBC from the AV sink device.
 39. The method of claim 38, wherein theUIBC capability is defined in a wireless fidelity (Wi-Fi) display (WFD)standard.
 40. The method of claim 38, wherein the AV source devicerequests the AV sink device to enable at least one UIBC from amongmultiple UIBCs provided by the sink device.
 41. The method of claim 40,wherein the AV source device receives a result of the enablement of theat least one UIBC from the AV sink device.
 42. A method of providing auser input back channel (UIBC) in an AV source device and an AV sinkdevice, the method comprising: receiving, at the AV sink device, a queryrelated to UIBC capabilities of the AV sink device from the AV sourcedevice; transmitting, by the AV sink device, a response related to theUIBC capabilities to the AV source device, receiving, a request toenable a desired UIBC, and enabling the desired UIBC to the AV sourcedevice.
 43. The method of claim 42, wherein the UIBC capability isdefined in a wireless fidelity (Wi-Fi) display (WFD) standard.
 44. Themethod of claim 42, wherein the AV sink device enables at least one UIBCfrom among multiple UIBCs provided by the AV sink device.
 45. Anaudio/video (AV) source device comprising: a controller which sets up anAV control session and an AV data session with an AV sink deviceaccording to a wireless fidelity (Wi-Fi) display (WFD) standard, and acommunicator which receives a user input from the AV sink device througha user input back channel (UIBC), wherein the UIBC from the AV sinkdevice is enabled using the AV control session.
 46. The method of claim45, wherein the user input comprises at least one of a function code forcontrolling the AV source device, pointing information, and characterdata.
 47. The method of claim 45, wherein the AV control session is areal-time streaming protocol (RTSP) session.
 48. An audio/video (AV)sink device comprising: a controller which sets up an AV control sessionand an AV data session with an AV source device according to a wirelessfidelity (Wi-Fi) display (WFD) standard, and a communicator whichtransmits a user input to the AV source device through a user input backchannel (UIBC), wherein the UIBC to the AV source device is enabledusing the AV control session.
 49. The method of claim 48, wherein theuser input comprises at least one of a function code for controlling theAV source device, pointing information, and character data.
 50. Themethod of claim 48, wherein the AV control session is a real-timestreaming protocol (RTSP) session.